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Major  Goals:  The  research  supported  by  this  ARO  grant  has  focused  on  the  control  of  continuous  time  stochastic 
systems  with  noise  that  is  Brownian  motions  or  fractional  Brownian  motions,  the  control  of  discrete  time  stochastic 
systems  with  arbitrary  correlated  noise  and  stochastic  differential  games.  In  modeling  physical  systems  the 
perturbations  or  the  unmodeled  dynamics  are  typically  represented  by  an  addditive  noise  perturbation  of  the 
mathematical  model.  Such  modeling  has  been  quite  effective  in  a  variety  of  physical  systems.  Some  important 
examples  are  space  exploration  and  telecommunications.  Historically  the  continuous  noise  has  been  modeled  by  a 
Brownian  motion  which  was  identified  in  the  physics  literature  in  the  beginning  of  the  twentieth  century  by  Einstein 
and  Smoluchowski.  However  based  on  empirical  data  from  many  physical  phenomena  it  has  been  verified  that 
other  noise  models  are  often  required.  One  family  of  stochastic  models  that  have  been  identified  empirically  is  the 
family  of  fractional  Brownian  motions.  The  fractional  Brownian  motions  are  a  family  of  centered  Gaussian 
processes  indexed  by  the  H8rst  parameter,  $H  \in  (0,1)$.  The  usefulness  of  these  processes  has  been  verified 
from  empirical  data  as  appropriate  for  models  of  rainfall,  turbulence,  economic  data,  cognition,  telecommunications, 
and  epileptic  seizures.  Most  physical  systems  require  stochastic  models.  Furthermore  most  physical  systems  are 
controlled.  The  strategies  can  be  represented  by  one  agent  or  by  multi-agents.  In  the  former  case  the  problem  can 
be  posed  as  stochastic  control  and  in  the  latter  case  the  problem  can  be  posed  as  stochastic  differential  games. 

The  game  problems  have  two  or  more  competing  agents  and  a  payoff  functional.  These  problems  arise  in  natural 
resources  allocation,  financial  systems  and  warfare.  The  stochastic  control  and  game  problems  are  formulated 
with  a  cost  functional  or  a  payoff  functional  to  optimize  and  can  evolve  in  a  finite  time  horizon  or  an  infinite  time 
horizon.  In  the  infinite  time  horizon  setting  the  functional  can  have  a  discount  factor  or  be  a  long  run  average 
(ergodic)  criteria.  Often  the  complete  system  state  is  not  available  to  the  controller  so  the  system  is  termed  partially 
observed.  In  this  case  the  state  of  the  system  has  to  be  estimated.  The  cost  functionals  and  payoff  functionals  are 
typically  quadratic  in  the  state  and  the  strategies  or  exponential  quadratic  to  allow  for  explicit  optimal  strategies. 
Many  stochastic  systems  require  modeling  as  stochastic  partial  differential  equations  so  some  control  problems 
and  games  are  formulated  in  this  setting.  Typical  partial  differential  equations  are  obtained  from  the  heat  equation 
or  the  wave  equation  which  can  be  used  to  model  many  distributed  systems.  The  stochastic  partial  differential 
equations  are  described  by  stochastic  equations  in  an  infinite  dimensional  Hilbert  space.  These  stochastic  partial 
differential  equations  can  be  driven  by  Brownian  motions  or  fractional  Brownian  motions.  The  control  or  game 
problems  in  this  infinite  dimensional  setting  have  usually  quadratic  or  exponential  quadratic  cost  or  payoff 
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functionals  and  these  results  have  been  investigated.  These  optimization  problems  can  be  solved  explicitly  for 
optimal  strategies.  Often  stochastic  systems  evolve  in  nonlinear  spaces  that  have  some  differential  geometric 
description.  Some  examples  of  these  spaces  that  are  called  symmetric  spaces  are  unit  spheres  or  open  unit  balls 
in  arbitrary  dimensional  Euclidean  spaces.  These  problems  have  been  investigated.  With  suitable  cost  criteria 
these  problems  can  be  explicitly  solved  for  both  control  and  game  problems  and  this  has  been  done.  Thus  finite 
and  infinite  dimensional  stochastic  systems  with  a  variety  of  noise  processes  are  considered  to  solve  control  and 
differential  game  problems  in  both  continuous  and  discrete  time.  All  of  the  above  types  of  problems  have  been 
studied  with  the  support  of  this  grant. 

The  achievement  of  these  goals  can  provide  some  major  contributions  to  the  scientific  base  of  the  United  States. 
Some  applications  of  these  results  can  be  important  for  contributing  to  important  applications  that  can  contribute  to 
the  industrial  development  of  the  United  States. 
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1  Accomplishments 

The  goals  proposed  for  this  study  have  been  largely  achieved  and  in  some 
cases  extended  beyond  the  problems  proposed.  A  direct  method  for  solving 
stochastic  control  and  stochastic  differential  games  has  been  developed  so 
that  solutions  of  control  or  game  problems  are  obtained  without  the  require¬ 
ments  of  solving  nonlinear  partial  differential  equations  (Hamilton-  Jacobi- 
Bellman  or  Hamilton-Jacobi-Isaacs  equations)  or  using  a  stochastic  maxi¬ 
mum  principle  with  backward  stochastic  differential  equations.  This  direct 
method  can  be  used  to  solve  both  continuous  and  discrete  time  control  prob¬ 
lems. 

One  type  of  problem  that  has  been  solved  is  a  stochastic  differential  game 
with  a  general  square  integrable  noise  process  and  a  quadratic  payoff  that  is 
described  now. 

dX(t)  =  AX(t)dt  +  BU(t)dt  +  CV(t)dt  +  FdW{t)  (1) 

X(0)  =  XQ 

where  X0  G  Rn  is  not  random,  X(t)  G  Rn,  A  G  £(Rn,R"),f?  G  £(Rm,Rn), 
U(t)  G  Rm,  U  eU,C  G  £(RP,  R"),  V{t)  G  Rp,  V  G  V,  and  F  G  £(R*,Rn). 
The  terms  U  and  V  denote  the  control  actions  of  the  two  players.  The  posi¬ 
tive  integers  (■ m,n,p,q )  are  arbitrary.  The  process  (W(t),t  >  0)  is  a  square 
integrable  stochastic  process  with  continuous  sample  paths  that  is  defined  on 
the  probability  space  (Q,  F,  P)  and  (. F(t),t  G  [0,  T])  is  the  filtration  for  W. 
The  family  of  admissible  strategies  for  U  is  U  and  for  V  is  V  and  they  are 
defined  as  follows 

U  =  {U  :  U  is  an  Rm- valued  process  that  is  progressively  measurable  with 

respect  to  (F(t),t  G  [0,  T])  such  that  U  G  L2([0,T])  a.s.} 

and 

V  =  {V  :  V  is  an  Rp- valued  process  that  is  progressively  measurable  with 
respect  to  ( F(t),t  G  [0,  T])  such  that  V  G  L2([0,T])  a.s.} 

The  cost  functional  J  is  a  quadratic  functional  of  X,  U,  and  V  that  is 
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given  by 


J°(U,V)  =  l-[f  (■ <QX(s),X(s)>  +  <RU(s),U(s)> 

-  <  SV(s),V(s)  >)ds+  <  MX(T),X(T)  >} 

J(U,V)  =  E  [J°(U,V)]  (2) 

where  Q  G  £(Rn,  Rn),  R  G  £(Rm,Rm),5e  G  £(Rn,Rn),  and 

Q  >  0,  R  >  0,  S  >  0,  and  M  >  0  are  symmetric  linear  transformations. 

The  following  theorem  provides  an  explicit  solution  to  this  noncooperative 
two  person  linear  quadratic  game  with  a  general  noise  process,  W.  It  seems 
that  there  are  no  other  results  available  for  these  games  when  W  is  an  arbi¬ 
trary  stochastic  process  with  continuous  sample  paths. 

Theorem  1.1.  The  two  person  zero  sum  stochastic  differential  game  given  by 
(??)  and  (??)  has  optimal  admissible  strategies  for  the  two  players,  denoted 


U*  and  V* ,  given  by 

U*(t )  =  -R-1(BTP(t)X(t)  +  BT(j>(t))  (3) 

V*(t)  =  S~1(CTP(t)X(t)  +  CTj)(t))  (4) 

where  ( P(t),t  G  [0,  T])  is  the  unique  positive  solution  of  the  following  equation 
dp 

=  QpPA  +  ATP-P{BR~lBT-CS-1CT)P  (5) 

P(T)  =  M  (6) 

arid  it  is  assumed  that  BR~1BT  —  CS~lCT  >  0  and  (</>(£),£  G  [0,  T])  is  the 
solution  of  the  following  linear  stochastic  equation 

d(j>(t)  =  ~[(AT  -  P(t)BR~lBT  +  P(f)CS~1CT)(t)dt  (7) 

+P{t)FdW{t)\ 

m  =  o  (8) 

and 

0(t)  =  E[0(t)|  P(t)}  (9) 


This  problem  has  been  generalized  in  various  ways.  The  linear  system  has 
been  generalized  to  an  infinite  dimensional  Hilbert  space  to  model  stochastic 
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partial  differential  equations  where  the  operator  A  is  the  generator  of  a  Cq 
semigroup  and  the  noise  is  a  cylindrical  fractional  Brownian  motion. 

dX(t)  =  AX(t)dt  +  BU(t)dt  +  CV(t)dt  +  $dW(t)  (10) 

X(0)  =  x 

where  X(t)  G  H  for  t  G  [0,  T\,  Xq  G  H,  %  is  a  real,  separable,  infinite 
dimensional  Hilbert  space,  A  :  Da  C  R  — >  'H  is  a  linear  and  (in  general) 
unbounded  operator  that  is  the  infinitesimal  generator  of  a  strongly  contin¬ 
uous  semigroup  ( S(t),t  >  0),  U(t)  G  U,V(t)  G  V  for  Hilbert  spaces  U  and 
V.  £(77i,772)  is  the  family  of  bounded  linear  operators  from  /H\  to  "H2-  The 
process  (W(t),t  G  [0,T])  is  a  standard  cylindrical  fractional  Brownian  mo¬ 
tion  in  H  with  the  Hurst  parameter  H  G  (|,1)  fixed  and  it  is  defined  on 
the  probability  space  (f^T7,  P)  and  {Tit)  A  G  [0,  T])  is  the  filtration  for  W 
on  this  probability  space.  The  linear  operator  A  is  usually  the  infinitesimal 
generator  of  an  analytic  semigroup  on  R  so  that  for  some  (3  >  0  the  operator 
— A  +  $1  is  strictly  positive  so  that  the  fractional  powers  (— A  +  /3/)7  and 
(— H*+/§/)7  and  the  spaces  D\  =  D((— H+/3/)7)  and  D\, *  =  £>((— H*+/3/)7) 
with  the  graph  norm  topology  for  7  G  R  can  be  dehned.  The  linear  space 
D(-)  denotes  the  domain  of  ■.  The  linear  operators  B,  C  and  <f>  and  the 
family  of  admissible  strategies,  (Ua,Va),  for  the  two  players  satisfy  natural 
conditions. 

An  ergodic  payoff  is  considered. 

The  payoff  for  a  T  >  0  is 

4(U,V)  =  i[/  (<QX(t),X(t)  (11) 

+  <  RU(t),  U(t)  >  -  <  SV(t ),  V(t)  >)dt ] 

JT(U,V)  =  E  J%(U,V)  (12) 

The  family  of  admissible  strategies  for  U  is  U  and  for  V  is  V  and  they  are 
dehned  as  follows 

U  —  {U  :  U  is  an  Rm-valued  process  that  is  (,F(t),t  G  [0,oo))  progres¬ 
sively  measurable  such  that  U  G  L2([0,T])  a.s.  for  each  T  >  0} 
and 

V  =  {V  :  V  is  an  Revalued  process  that  is  G  [0,oo))  progressively 
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measurable  process  such  that  V  £  L2([0,T])  a.s.  for  each  T  >  0} 


where  (F(t),t  £  [0, oo))  is  the  natural  filtration  for  (W(t),t  >  0).  The 
expected  long  run  average  (ergodic)  cost  for  the  stochastic  game  is 

V )  =  Urn  supt^MU,  V ) 

The  formal  Riccati  equation  for  this  stochastic  game  which  can  be  precisely 
defined  is 

dP 

—  =  A*P  +  PA  —  P(BR~1B*  —  CS~lC*)P  +  Q  (13) 
P(0)  =  G 

Theorem  1.2.  Let  some  natural  assumptions  be  satisfied.  For  the  stochastic 
differential  game  given  by  (??)  and  (??)  and  the  admissible  strategies  Ua  and 


Va  there  are  optimal  strategies,  U*  and  V* ,  given  by 

U*(t )  =  -R-1P*(P(t)X(t)  +  ^(t))  (14) 

V*(t)  =  p-1C'*(P(t)X(t)  +  ^(t))  (15) 

where 

d<p(t)  =  -[(A*  -  P(t)BR~lB*  +  P(t)CS-lC*)(pdt  (16) 

+P(t)<S>dW(t)] 

<P(T)  =  0  (17) 

and 

<p(t)  =  E[<p(t)\F(t)]  (18) 


The  optimal  payoff  for  this  Nash  equilibrium  can  be  computed  directly. 

Some  nonlinear  stochastic  differential  games  can  be  explicitly  solved.  The 
following  game  evolves  in  the  unit  sphere  in  Euclidean  three-space.  The 
stochastic  differential  game  is  described  by  the  following  equation  which  de¬ 
scribes  the  distance  of  the  process,  (Y(t),t  £  [0,  T]),  from  a  point  on  the 
sphere  denoted  o  which  is  called  the  origin  for  the  differential  game. 


dX(t) 

1  X  (t) 

=  -cot  —  dt  +  bU  ( t)dt  +  cV  ( t)dt 

(19) 

+  dB(t) 

X(0) 

=  x0 

(20) 
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where  Y(t)  G  S2  \  Aa,  where  A0  is  the  antipodal  manifold,  X(t)  =  |Y(£)|, 
{B(t),  t  G  [0,  T})  is  a  real- valued  standard  Brownian  motion  for  a  fixed  T  >  0, 
A"o  G  (0,  L)  is  a  constant  and  L  is  the  distance  from  o  to  Aa.  The  Brownian 
motion  is  defined  on  the  complete  probability  space  (hi,  T,  P)  and  {Tit)  A  G 
[0,  T])  is  the  filtration  for  the  Brownian  motion  B.  The  terms  {b,  c)  are 
nonzero  real  numbers.  An  assumption  on  the  relative  size  of  these  two  real 
numbers  is  made  subsequently.  The  family  of  admissible  control  strategies 
for  U  is  U  and  for  V  is  V  and  they  are  defined  as  follows 
U  =  {U  :  U  is  an  ]Rm- valued  process  that  is  progressively  measurable  with 
respect  to  {T{t),t  G  [0,  T])  such  that  U  G  L2([0,T])  a.s.} 
and 

V  =  {V  :  V  is  an  IRA  valued  process  that  is  progressively  measurable  with 
respect  to  {T{t),t  G  [0,  T])  such  that  V  G  L2([0,T])  a.s.} 

If  U{t)  and  V (t)  are  suitably  smooth  functions  of  X(t),  then  (X(t),t  G  [0,T]) 
is  a  Markov  process  with  the  infinitesimal  generator 


1  d2 

2  dr 2 


1  r  d  d  d 


(21) 


The  payoff  for  the  stochastic  differential  game  with  the  control  strategies  U 
and  V  is  denoted  Jt(U,  V)  that  is  described  as  follows 


4(u,v) 

=  [T{a  sin2*-®  +  U2(t)cos2X® 

Jo  4  4 

—  V2{t)cos2*®  )dt 

(22) 

Jt(U,V) 

=  E  4(u,v) 

(23) 

The  following  scalar  Riccati  and  linear  equations  are  used  in  the  solution  of 
the  control  problem. 


dgjt) 

dt 

g(T ) 
dh(t) 
dt 

h(T) 


3  +1  2(62  2) 

8  16  ' 


=  0 


169 


=  0 


(24) 

(25) 

(26) 
(27) 


Theorem  1.3.  Let  b2  >  c2  >  0.  The  stochastic  differential  game  described 
by  (??)  and  (??)  has  optimal  control  strategies,  ( U*,V *),  for  the  two  players 
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that  are  given  by 


U*(t )  = 

V*(t)  =  ^ g(t)tan 

where  t  E  [0,T]  arid,  g  is  the  positive  solution  of  the  scalar  Riccati  equation. 
The  value  of  the  game  is 

JT(U*,  V*)  =  g(0)sm2^  +  h( 0) 

where  h  is  given  by  (??). 

A  similar  approach  can  be  used  to  verify  optimal  control  strategies  for 
some  infinite  time  horizon  problems  with  a  long  run  average  payoff  for  the 
stochastic  differential  game.  In  this  case  the  payoff  is 

1 

Joo(U,  V )  =  Urn  supt^oo^jt{.U,  V ) 

where  Jt  is  given  above.  The  payoff  function  for  these  game  problems  has 
the  important  property  that  sin2 1  is  an  eigenfunction  of  the  radial  part  of 
the  Laplacian  for  S2.  Other  payoff  functionals  can  be  defined  by  choosing 
other  eigenfunctions  of  the  radial  part  of  the  Laplacian.  Similarly  stochastic 
differential  games  in  the  n-sphere  and  other  compact  rank  one  symmetric 
spaces  can  be  formulated  and  explicitly  solved. 

The  following  control  problem  is  for  a  linear  equation  in  a  Hilbert  space 
with  an  additive  fractional  Brownian  motion  and  an  ergodic  quadratic  cost. 
An  infinite  horizon  control  problem  is  described  by  a  long  term  average  or 
ergodic  quadratic  cost  functional. 

dX(t)  =  ( AX(t)+Bu(t))dt  +  dBH(t )  (28) 

X(0)  =  x  (29) 

where  x  E  V,  X(t)  G  V,  V  is  an  infinite  dimensional  real  separable  Hilbert 
space  with  inner  product  <  •,  •  >  and  norm  j  •  j.  The  process  (. Bn(t),t  >  0)  is 
a  V  -valued  fractional  Brownian  motion  with  the  Hurst  parameter  He  (|,  1) 
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and  having  the  incremental  covariance  Q  where  Q  is  trace  class  ( Tr(Q )  <  oo) 
so  that 

E  <  BH(t),x  ><  BH(s),y  >=  ^  <  Qx,  y  >  ( t2H  +  s2H  -  1 1  -  s\2H).  (30) 

for  x,y  G  V.  The  operator  A  :  Dom(A)  — >  V  with  Dom(A)  C  V  is  a 
linear,  densely  defined  operator  on  V  which  is  the  infinitesimal  generator  of 
a  strongly  continuous  semigroup  ( S(t),t  >  0).  Let  U  —  (U,  <  ■,  •  >u,  \  •  |jj) 
be  another  Hilbert  space,  the  state  space  of  controls,  and  assume  that  B  € 
jC(U,V).  Furthermore  consider  the  family  of  admissible  controls,  VI,  defined 
as  follows 


U  =  {u  \  R+  x  Q  — y  U,  u  is  progressively  measurable, 

rT 


E 


\u(t)\fjdt  <  oo  for  all  T  >  0} 


The  solution  of  the  equation  (??)  is  defined  as  the  mild  solution,  that  is, 


X(t)  =  S(t)x  +  f  S(t  —  s)Bu(t)dt  T  f  S(t  —  s)dBH(t)  (31) 

Jo  Jo 

for  t  >  0  and  it  is  known  that  with  the  above  assumptions  there  is  one  and 
only  one  V-  continuous  solution  to  (??)  .  Now  the  cost  functional  is  defined 
for  the  control  problem.  Let  Jt  be  given  as  follows 

JT(x,u )  :=  ^  [  (|LA"(s)|2+  <  Ru(s),u(s )  >u)ds  (32) 

^  Jo 

where  L  G  C,{V),  R  G  C,{U),  R  is  self-adjoint  and  invertible.  The  control 
problem  is  to  minimize  the  following  ergodic  cost 

lim  supt^oo^^Jt(x,u).  (33) 

Theorem  1.4.  Let  detectability  and  stabilizability  conditions  be  satisfied  and 
let  u  GW  be  a  control  satisfying 

UmT^ oo^E  <  PXU(T ),  XU(T)  >=  0  (34) 


where  (XU(T),T  G  [0,  oo))  is  the  solution  to  (??)  with  the  control  u  G  W. 
Then 


lim  supr^oo— EJt(x,m)  >  Joo 


(35) 
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where 


-|  rT 

Joo  :=  lim  supr^—E  j  \R^ B*V(s)\^ds  (36) 

/»oo 

+  /  Tr(QP$(t))(/>H(r)dr  (37) 

Jo 

for  each  x  G  V  where  <f>(r)  =  H(2H  —  l)\r\2H~2,r  G  R.  Moreover,  the 
feedback  control  u(t )  =  —R~lB*(PXu(s)  +  V(s))  is  admissible,  satisfies  the 
condition  (??)  and 

1 

lim  supt^oot^Jt{x,u)  =  Joo  (38) 

for  each  x  G  V .  Thus  u  is  an  optimal  ergodic  control  and  is  the  optimal 
cost  for  the  ergodic  control  problem  ( (??)-(??) ). 

Consider  the  following  controlled  linear  stochastic  system  with  a  general 
noise  process 


dX(t)  =  AX(t)dt  +  CU(t)dt  +  dB(t)  (39) 

X(0)  =  X0  (40) 

where  Xq  G  Rn  is  not  random,  X(t)  G  Rn,A  G  £(R",R"),C  G  £(Rm,Rn), 
U(t)  G  Rm,  £(Rfc,Rz)  denotes  the  family  of  linear  transformations  from  Rk 
to  E/,  U  GW,  (. B(t),t  G  [0,  T])  is  an  Rn-valued  zero  mean,  square  integrablc 
process  with  continuous  sample  paths  with  5(0)  =  0  and  this  process  is 
dehned  on  the  complete  probability  space  (17,  T ,  P)  and  T  >  0  is  fixed. 

The  family  of  nonadapted  admissible  controls,  14,  is 
U  =  {U  :  U  is  an  Rm-valued  process  such  that  U  G  £2([0,T])  a.s.} 

Let  (Jr(t),t  G  [0,  £])  be  the  filtration  of  ( B(t),t  G  [0,T]).  The  family  of 
adapted,  admissible  controls,  Ua,  is 

Ua  —  {U  :  U  is  an  Rm- valued  (JF{t),t  G  [0,T])  progressively  measurable 
process  such  that  U  G  £2([0,T])  a.s.} 

The  cost  functional  J  is  a  quadratic  functional  of  X  and  U  that  is  given  by 

J(U )  =  \e[  f  <  QX(s),X(s)  >  +  <  RU(s),  U(s)  >  ds } 

2  Jo 

+  ^E  <  MX(T),X(T)  > 


(41) 


where  Q  G  £(Rn,Rn),P  G  £(Rm,Rm),M  G  £(Rn,Rn),  Q  >  0,  R  >  0 
and  M  >  0  are  symmetric  linear  transformations  and  <  >  denotes  the 

canonical  Euclidean  inner  product  on  the  Euclidean  space  of  the  appropriate 
dimension.  The  dependence  of  J  on  the  initial  condition  X0  is  suppressed 
for  notational  convenience. 


Theorem  1.5.  For  the  optimal  control  problem  (??)  and  (??)  and  the  family 
of  admissible,  nonadapted  controls,  U,  there  is  an  optimal  control  U*  that  can 
be  expressed  as 

U*(t)  =  -R-1CT(P(t)X(t)  +  W(t))  (42) 


where  (P(t),t  G  [0,  T])  is  the  unique  symmetric  positive  definite  solution  of 
the  Riccati  equation 


—  =  -PA-  AtP  +  PCR~1CtP  -Q 
dt 

P(T)  =  M 


and  ( W(t),t  G  [0,  T])  is  the  process  that  satisfies 


W(t)  = 


$r(s,t)P(s)dB(s ) 


(43) 

(44) 


(45) 


and  B  is  the  process  in  (??)  and  <f>p  is  the  fundamental  solution  of  the  matrix 
equation 


d<hp(s,  t ) 

dt 

®p(s,s) 


-( AT  -  P(t)CR-lCT)$p(s ,  t) 

I 


(46) 

(47) 


Corollary  1.6.  Let  ( B(t),t  G  [0,  T])  in  (??)  be  a  standard  fractional  Brown¬ 
ian  motion  with  a  fixed  Hurst  parameter  H  G  (0, 1).  For  the  optimal  control 
problem  given  by  (??)  and  (??)  and  the  family  of  admissible  nonadapted 
controls  U  an  optimal  control  U*  is  given  by 

U*(t)  =  R~lCT  (P(t)X(t)  +  W(t))  (48) 


where  ( P(t),t  G  [0,  T])  is  the  unique  positive  definite  symmetric  solution  of 
(??)  and  ( W(t),t  G  [0,  T])  is  the  process  that  satisfies 


$P(s,t)P(s)dB(s ) 


(49) 
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(50) 


and  is  the  solution  of  (??).  For  H  G  (|,  1)  the  optimal  cost  is 
J(U')  =  t<P(0)X0,X0> 

r  f  j (  tr(P(r)$p(r,  t)CR-1CTiP(s,  t)P( ») 

x4>h(s  —  r)drdsdt  +  f  f  tr($p(s,t)P(s )) 

Jo  Js 

xJ>h(s  —  t)dsdt 

where  4>h(s)  =  H(2H  —  l)\s\2H~2. 

Now  a  partially  observed  control  problem  is  described  with  a  risk  sensi¬ 
tive  cost  functional.  Initially  the  system  and  the  observation  equations  are 
described.  The  equation  for  the  system  process  X  is  given  by 

dX(t)  =  ( AX(t)  +  CU(t))dt  +  FdB(t )  (51) 

X(0)  =  X0 

where  X0  is  a  constant  vector  in  R",  X(t)  G  Rn,  U(t)  G  Rm,  A  G  L(Rn,  Rn),  C  G 
L(Rm,  Rn),  F  G  L(Rn,  Rn)  and  ( B(t),t  >  0)  is  an  Rn- valued  standard  Brow¬ 
nian  motion.  The  process  B  is  defined  on  the  complete  probability  space 

(ax,p). 

The  observation  process  (Y(t),t  G  [0,  T])  satisfies  the  following  stochastic 
equation 


dY(t)  =  HX(t)dt  +  GdV(t)  (52) 

y(o)  =  0 

where  Y(t)  G  L(RP),  H  G  L(Rn.Rp),  G  G  L(RP,RP)  is  invertible  and 
( V(t),t  >  0)  is  an  Rp- valued  standard  Brownian  motion  that  is  also  defined 
on  (Q,  X,  P).  It  is  assumed  that  the  processes  B  and  V  are  independent.  Let 
(G(t),t  G  [0,  T])  be  the  natural  filtration  for  the  process  (Y(t),t  G  [0,T])  on 
(hi,  X}  P).  The  family  of  admissible  controls,  U ,  is  defined  as 
U  =  {U  :  IJ  is  an  Rm- valued  G  [0,T])  progressively 

measurable  process  such  that  U  G  L2([0,T])  a.s.} 

The  cost,  </(•),  is  an  exponential  quadratic  functional  of  the  state  and  the 
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(53) 


control  that  is  given  as  follows 

PT 

J(U)  =  pEexp[^  /  (<QX(s),X(s)> 

4  Jo 

+  <  RU(s),  U(s)  >)ds  +  ^<  MX(T),X(T)  >} 

where  Q  G  £(Rn,Rn),  R  G  £(Rm,Rm)  and  M  G  £(Rn,Rn)  are  symmetric 
linear  transformations,  such  that  Q>0,R>0,  M>0  and  /i  is  hxed.  For 
the  verihcation  of  an  optimal  control  in  this  paper  it  is  assumed  that  M  =  0. 
Some  remarks  are  made  later  about  this  restriction  and  how  to  eliminate  it. 
The  appropriate  estimation  equation,  often  called  the  information  filter,  is 
given  by 

dZ{t)  =  (A-  P(t)HTH  +  fiP(t)Q)Z(t)dt  (54) 

+CU(t)dt  +  P(t)HTdY(t ) 

Z(0)  =  X(0) 


and  (. P(t),t  G  [0,  T])  is  the  unique,  positive  symmetric  solution  of  the  follow¬ 
ing  Riccati  equation 

^  =  AP  +  PAT  (55) 

P(HtH  —  nQ  +  FFt)P 

P{  0)  =  0 


It  is  assumed  that  /i  is  chosen  to  satisfy  ( HT H  —  nQ  +  FFT )  >  0.  The 
process  (f*  PHT(dY'  —  HZds),Q(t),t  G  [0,  T])  is  a  Brownian  motion  by 
the  Riccati  equation  (??)  and  sn  absolute  continuity  result  It  follows  from 
the  results  for  the  information  filter  that  for  observation  measurable  actions 
on  the  exponential  quadratic  cost,  that  it  suffices  to  consider  the  process 
( Z(t),t  G  [0,  T])  because  this  process  is  the  minimizing  solution  of  the  best 
estimate  for  the  exponential  of  the  quadratic  form  in  X  formed  using  Q. 
Thus  the  control  for  (??)  is  a  function  of  the  process  Z.  This  estimate  Z  is 
given  as  follows 


z(-) 


arg  minhenE[/xexp(^ 
X(s)  -  h(s )  >  ds\G(t)} 


<Q(X(s)-h(s)), 


(56) 


where  Ft  is  the  family  of  square  integrable  G(-)  progressively  measurable 
processes  on  [0,  T] 
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Theorem  1.7.  For  the  control  problem  given  by  the  state  equation  (??),  the 
observation  equation  (??),  and  the  cost  functional  (??)  there  is  an  optimal 
control,  U* ,  from  the  family  of  admissible  controls,  U,  that  is  given  by 

U*(t )  =  -R~lCT  S(t)Z(t)  (57) 

where  ( S(t),t  E  [0,T])  is  the  unique  positive,  symmetric  solution  of  the  fol¬ 
lowing  Riccati  equation 

7  Q 

=  S(A  +  pPQ)  +  (AT +  pQP)S+Q  (58) 

-(S(CR~1CT  -  pPHT(GGT)~1HP)S 
S{T)  =  0 

Consider  a  financial  market  defined  on  a  complete  probability  space  (12,  T ,  P) 
with  filtration  F  =  T  <  oo,  satisfying  the  usual  conditions  and 

T  =  Pr-  Without  loss  of  generality  it  is  assumed  that  the  savings  account  is 
constant  and  identically  equal  to  one.  Moreover,  it  is  assumed  that  the  price 
X  of  the  underlying  asset  has  a  stochastic  volatility  given  by  a  function  of  a 
standard  fractional  Brownian  motion,  so  the  dynamics  of  X  is  given  by 

dX(t)  =  f(WH(t))g(t)X(t)  dW(t ),  (59) 

where  X(0)  is  a  positive  constant,  the  process  W  is  a  standard  Brownian 
motion,  WH  is  a  standard  fractional  Brownian  motion  with  the  Hurst  pa¬ 
rameter  He  (0, 1),  /  :  M  — *  M+  is  Borel  measurable  and  g  :  [0,  T }  — »  M+  is 
Borel  measurable  and  bounded. 


Theorem  1.8.  Lett  E  [0,  T\,  X  be  given  by  (??),  WH  the  fractional  Brown 


ian  motion  and  W,  W  be  correlated  Brownian  motions,  d\W,  Wj  =  p(t)dt 

with  a  measurable,  deterministic  function  p  :  [0,T]  — »  (—1,1).  The  random 
variable  Xt  has  a  probability  density  function  hxt  satisfying 


In  *  -  J‘  f(WH(u))g(u)p(u)dW(u)  +  1/2  f‘  f(W«(u))g*(u)du 


hx (t)(s)  =  E  -^—pl  — — 

sag  \  oh 

(60) 

where  s  >  0,  p  is  the  probability  density  of  a  standard  Gaussian  random 
variable  N{ 0, 1),  and 

°rH=  [  f'\WH{u))g2{u)(l  -  p2{u))du.  (61) 
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A  two-person  stochastic  differential  game  with  a  risk  sensitive  quadratic 
payoff  is  considered  now.  The  two  person  stochastic  differential  game  is 
described  by  the  following  linear  stochastic  differential  equation 

dX(t)  =  AX(t)dt  +  BU(t)dt  +  CV(t)dt  +  FdW(t)  (62) 
X(0)  =  X0  (63) 

where  X0  G  Rn  is  not  random,  X(t)  G  Rn,  A  G  £(Rn,R"),R  G  £(R,n,Rn), 
U(t)  G  Rm,  U  eU,C  G  £(RP,  Rn),  V(t)  G  Rp,  V  G  V,  and  F  G  £(R9,Rn). 
The  positive  integers  (■ m,n,p,q )  are  arbitrary.  The  process  (W(t),t  >  0) 
is  an  R9  valued  standard  Brownian  motion  that  is  defined  on  the  complete 
probability  space  (hi,  F,  P)  and  (F(t),t  G  [0,  T])  is  the  filtration  for  W.  The 
terms  U  and  V  denote  the  strategies  of  the  two  players  and  the  family  of 
admissible  strategies  for  U  is  U  and  for  V  is  V  and  these  families  are  defined 
as  follows 

U  =  {U  :  U  is  an  Rm -valued  process  that  is  progressively  measurable  with 

respect  to  (F(t),t  G  [0,  T])  such  that  U  G  L2([0,T])  a.s.} 

and 

V  =  {V  :  V  is  an  Rp- valued  process  that  is  progressively  measurable  with 
respect  to  (F(t),t  G  [0,  T])  such  that  V  G  L2([0,T])  a.s.} 

The  payoff  is  the  exponential  of  a  quadratic  functional  of  X,  U,  and  V 
that  is  given  by 

/T 

(<QX(s),X(s)>  (64) 

+  <  RU{s),  U(s)  >  -  <  SV(s),  V(s)  >)ds 
+  |  <  MX{T),X{T)  >] 

MU,V)  =  E  [J°(U,V)}  (65) 

where  Q  G  £(Rn,  Rn),  R  G  £(Rm,  Rm),  S  G  C(W,W),M  G  £(Rn,Rn),  and 
Q  >  0,  R  >  0,  S  >  0,  and  M  >  0  are  symmetric  linear  transformations  and 
li  ^  0  is  fixed.  An  assumption  on  the  possible  values  for  //  is  given  in  the 
following  theorem.  The  player  with  control  U  seeks  to  minimize  the  payoff 
J/t  while  the  player  with  control  V  seeks  to  maximize  the  payoff  J fl. 

Theorem  1.9.  The  two  person  zero  sum  stochastic  differential  game  de¬ 
scribed  by  (??)  and  (??)  has  a  Nash  equilibrium  using  the  optimal  admissible 


13 


control  strategies  for  the  two  players,  denoted  U*  and  V* ,  given  by 


U*(t )  =  -R~1BTP{t)X{t)  (66) 

V*(t)  =  S~lCTP(t)X(t )  (67) 

where  ( P(t),t  G  [0,T])  is  the  unique  positive  symmetric  solution  of  the  fol¬ 
lowing  Riccati  equation 

dP 

=  Q  +  PA  +  ATP  (68) 

-  P(BR~1Bt  -  CS~lCT  -  pFFT)P 
P(T )  =  M  (69) 

and  it  is  assumed  that  BR ~lBT  —  CS~lCT  —  pFFT  >  0.  The  optimal  payoff 

is 


fT 

MU*,V*)  =  pexp{^{<  P{ 0)X0,X0  >  +  /  tr(PFFT)dt )]  (70) 

^  Jo 

A  stochastic  differential  game  problem  that  is  formulated  and  solved  to 
control  the  roots  of  a  process  in  the  Lie  algebra  su( 3)  is  now  described.  Since 
SU( 3)  is  simply  connected,  this  game  problem  can  be  viewed  in  the  Lie  al¬ 
gebra,  su( 3).  The  group  SU( 3)  has  particular  interest  in  physics  because 
the  Gell-Mann  matrices  are  generators  for  SU (3)  that  mediate  Quantum 
Chromodynamics  (QCD)  which  is  also  known  as  the  Strong  Force.  In  theo¬ 
retical  physics  QCD  is  the  theory  of  strong  interactions  that  is  a  fundamental 
force  describing  the  interactions  between  quarks  and  gluons  which  comprise 
hadrons  such  as  the  proton,  neutron  and  pion.  This  theory  is  an  important 
part  of  the  Standard  Model  of  particle  physics. 

The  simply  connected  Lie  group  SU (3)  is  the  family  of  3  x  3  unitary  matrices 
with  determinant  one,  that  is,  g  e  577(3)  if  gg*  =  /,  det(g)  =  1.  This  Lie 
group  has  dimension  eight  as  a  real  manifold.  It  is  a  simple  Lie  group.  This 
Lie  group  has  rank  two,  that  is,  the  dimension  of  the  Cartan  subalgebra  is 
two. 

The  stochastic  differential  game  is  described  by  a  stochastic  differential 
equation  that  has  terms  from  the  strategies  of  the  two  players  and  terms 
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from  the  radial  part  of  the  Laplacian. 


dXiit) 

dX2(t) 

Mo) 

■MO) 


-(ccoth  Ad(t))  +  6  coth  — +  a 


+  aUi(t)dt  + /3Vi(t)dt  +  dBi(t) 
=  -  (c  coth  X2  ( t )  +  b  coth  — ^ 

+  aU2(t)dt  +  /3V2(t)dt  +  dB2(t) 


sinh  Xl  (t) 

cosh  X2  (t)  —  cosh  Xi  (t) 


sinh  X2(t) 

-|-  d - - — - - 

cosh  X\  (t)  —  cosh  X2(t) 


(71) 

)dt 


The  process  B2(t)),  t  G  [0,  T])  is  an  R2-valued  standard  Brown¬ 

ian  motion  that  is  defined  on  the  complete  probability  space  (hi,  X P)  and 
(X(t),t  G  [0,  T])  is  the  filtration  for  the  Brownian  motion  (Bi,B2),  xw  and 
x2q  are  constants  and  a,  fd  are  strictly  positive  constants.  Player  I  has  the 
control  pair  (U\,  U2)  and  player  II  has  the  control  pair  (V) ,  V2).  It  is  assumed 
that  the  positive  real  numbers  a,/3  satisfy  a2  —  fd2  >  0.  The  symmetry  of 
the  two  scalar  equations  for  X\  and  X2  is  inherited  from  the  coordinate  sym¬ 
metry  for  the  radial  part  of  the  Laplacian.  The  payoff  functional,  J(U,V), 
is 


J°(U,  V )  =  Asinh2  +  sinh2  +  (f/2(t)  -  V2(t))  cosh2 

Jo  2  2  2 

+(e22(i)-q2(i))cosh2^)<ft  (75) 

J(U,V)  =  E  J°(U,V)  (76) 


Theorem  1.10.  The  stochastic  differential  game  given  by  (??),  (??),  and 
(??)  has  the  following  optimal  strategies,  (U*,V*),  that  form  a  Nash  equilib¬ 
rium 

Uf(t)  =  —^ag(t)  tanh  (77) 

U2(t)  =  ~ag{t)  tanh— ^  (78) 

Vf{t)  =  ^/3g(t)  tanh  (79) 

V2*(t)  =  ^/3g(t)  tanh  7^^  (80) 
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The  optimal  payoff  is 

J(U*,  V*)  =  g(0)(sinh2  ^  +  sinh2  +  h( 0)  (81) 

A  control  problem  is  solved  for  a  stochastic  evolution  equation  with  a  state 
dependent  noise  process.  The  noise  can  be  a  fractional  Brownian  motion  for 
the  Hurst  parameter  in  the  interval  (|,1)  or  some  other  noise  processes. 

The  controls  are  restricted  to  linear  state  feedback.  Consider  the  stochastic 
evolution  equation 

dX(t)  =  (A(t)X(t)  +  B(t)K(t)X(t))dt  +  o{t)X{t)db{t) 

X  (0)  =  x0 

in  a  separable  real  Hilbert  space  V  =  (V,  \  ■  |, ,(•,•))  where  ( b(t),t  >  0)  is 
a  real- valued  Gauss- Volterra  noise  that  is  described  below,  ( A(t),t  >  0) 
is  a  family  of  closed,  (in  general)  unbounded,  operators  on  V  such  that 
Dom(A(f))  =  Dom(A(0))  for  each  t  G  M+,  and  Dom(A*(f))  =  Dom(A*(0)), 
that  generates  a  strongly  continuous  evolution  operator  ( U0(t,s)R ,  0  <  s  < 
t  <  oo).  Furthermore,  denoting  by  Cs([a,  6]),  C(Y Y2))  the  family  of  strongly 
continuous  mappings  [a,  b]  — >  C(YX ,  Y2)  where  Y\ ,  Y2  are  Hilbert  spaces,  B  e 
C',(M+,£(H,H))  and  K  e  C'S(R+,  £(V,  U)),  where  (U  =  U,(-,  -)v,\  •  \v)  is 
another  Hilbert  space;  the  process  u(t)  =  K(t)X(t)  is  described  as  a  linear 
feedback  control  of  the  system  and  some  linear-quadratic  control  problems 
are  studied  in  the  subsequent  sections.  Finally,  a  is  a  continuous  real- valued 
function. 

Some  details  concerning  the  real- valued  driving  process  ( b(t),t  >  0)  are 
given  now.  The  process  ( b(t),t  >  0)  is  a  Gauss- Volterra  process,  which  is 
described  by  the  covariance 

/•min  (t,s) 

R(t,s)  =  Eb(t)b(s)  :=  /  K(t,r)K(s,r)dr ,  (82) 

Jo 

where  the  kernel  K  :  — y  M  satisfies  some  conditions. 

The  admissible  controls  are  of  the  state  feedback  form  u(t)  =  K{f)X{t) 
where  K  e  Cs([0,  T],  £(V,  U)).  The  cost  functional  to  be  minimized  is 

Jt(K)  :=  E  [  (\L(f)X(f)\2  +  (R(t)K(t)X(t),K(t)X(t))u)dt  +  E{GX(T),X(T))(83) 
Jo 
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where  L  G  Cs([0,  T\,C(V))  and  G  =  G*,G  e  C(V),  G  >  0,  R  G  <^([0,  T),C(U)) 
is  such  that  R{t )  =  R*(t),  and  for  some  70  >  0,  ( R(t)u,u)u  >  7o|m|^,  u  G  U, 
t  G  [0,  T], 

The  operator  Riccati  differential  equation  associated  with  this  control 
problem  is 

P(t)  +  A*(t)P(t)  +  P(t)A(t)  -  P{t)B(t)R^(t)B*(t)P{t)  +  L*(t)L(t)  (84) 

—  a(t)P(t )  =  0,  t  G  (0,  T], 

P(T)  =  G. 

Note  that  this  Riccati  equation  is  different  from  the  Riccati  equation  for  a 
linear-quadratic  control  problem. 

The  following  result  solves  the  finite  time  horizon  problem. 

Theorem  1.11.  Let  conditions  on  the  noise  and  the  Riccati  equation  be 
satisfied.  Then  the  feedback  control  u(t)  =  —R~lB*{t)P{t)X{t)  is  optimal 
for  the  control  problem  that  is,  the  operator  function  K(t )  =  R~1(t)B*(t)P(t) 
minimizes  Jt  on  the  space  of  all  K  G  Cs([0,  T],  C(V,  U)).  The  optimal  cost 
is 


Jt{K)  =  (P(  0)xo,xo). 


(85) 
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